Medical Content Searching, Retrieving, and Sharing Over the Internet : Lessons Learned From the mEducator Through a Scenario-Based Evaluation

Background: The mEducator Best Practice Network (BPN) implemented and extended standards and reference models in e-learning to develop innovative frameworks as well as solutions that enable specialized state-of-the-art medical educational content to be discovered, retrieved, shared, and re-purposed across European Institutions, targeting medical students, doctors, educators and health care professionals. Scenario-based evaluation for usability testing, complemented with data from online questionnaires and field notes of users' performance, was designed and utilized for the evaluation of these solutions. Objective: The objective of this work is twofold: (1) to describe one instantiation of the mEducator BPN solutions (mEducator3.0 - "MEdical Education LINnked Arena" MELINA+) with a focus on the metadata schema used, as well as on other aspects of the system that pertain to usability and acceptance, and (2) to present evaluation results on the suitability of the proposed metadata schema for searching, retrieving, and sharing of medical content and with respect to the overall usability and acceptance of the system from the target users. Methods: A comprehensive evaluation methodology framework was developed and applied to four case studies, which were conducted in four different countries (ie, Greece, Cyprus, Bulgaria and Romania), with a total of 126 participants. In these case studies, scenarios referring to creating, sharing, and retrieving medical educational content using mEducator3.0 were used. The data were collected through two online questionnaires, consisting of 36 closed-ended questions and two open-ended questions that referred to mEducator 3.0 and through the use of field notes during scenario-based evaluations. Results: The main findings of the study showed that even though the informational needs of the mEducator target groups were addressed to a satisfactory extent and the metadata schema supported content creation, sharing, and retrieval from an end-user perspective, users faced difficulties in achieving a shared understanding of the meaning of some metadata fields and in correctly managing the intellectual property rights of repurposed content. Conclusions: The results of this evaluation impact researchers, medical professionals, and designers interested in using similar systems for educational content sharing in medical and other domains. Recommendations on how to improve the search, retrieval, identification, and obtaining of medical resources are provided, by addressing issues of content description metadata, content description procedures, and intellectual property rights for re-purposed content.Peer reviewe

The instructional design of the Virtual Telemedicine: A simulation-based serious game in health care

This paper is a work-in-progress report on the instructional design of Virtual Telemedicine, a simulation-based serious game in health-care. The problem that this game addresses is the need to train practicing doctors and medical students for problem-solving in real-life scenarios through a telemedicine system. In medical emergency situations a doctor is asked to make important decisions for the treatment of a patient within minutes or even seconds. His actions result in saving or losing a patient’s life. The importance of the development of this game lies on the opportunities it provides to students for practicing their patient-treatment skills in a safe and controlled e-learning environment that simulates real-life conditions but avoids the risks associated with dealing with real patients. The game makes use of data from an electrocardiogram (ECG) and its overall learning goal is for practicing doctors or medical students to learn how to respond to a medical emergency situation to treat the symptoms of a virtual patient by taking specific actions and examining their results based on immediate feedback provided by both the system and the virtual patient. Three scenarios are currently developed in the game: a) Precordial pain, in which the patient faces an acute coronary syndrome, b) Palpitations, in which the patient experiences cardiac arrhythmias and c) Syncope, in which the patient is diagnosed with an atrioventricular block. The paper reports on the instructional design objectives of the game, describes the prototype scenario and outlines the changes made as part of the game’s redesign and development to increase its pedagogical value. Some of the changes that were made to the beta-version of the game included the addition of the overall goal of the game, the objectives and context for each scenario, the patient’s history, as well as specific instructions to scaffold the user in making a choice among several treatment options. Moreover, instant feedback is now provided to the user as he receives a clearly visible positive or negative score for every one of his actions. In addition, the user can track his score while playing the game. Finally, a detailed report is provided at the end of the game outlining the user’s choices, clearly indicating which of them were successful and including the total amount of time he required to solve the problem and save the patient. The paper concludes with the description of the evaluation methodology of the Virtual Telemedicine game

Virtual Telemedicine: A serious game to develop problem solving skills in medical education

Η συγκεκριμένη δημοσίευση συνεδρίου βρίσκεται στο βιβλίο με τίτλο: EDULEARN12 ProceedingsSerious games have the potential to be an important teaching tool for both formal and informal education because of their affordances of interactivity and motivation, engaging users, providing a platform for active learning, being customized to learners, providing immediate feedback and including immersive activities. Serious games for medical education is a growing domain. Across the healthcare sector, there is growing interest in improving and sustaining interaction and engagement using game technologies. Game environments provide a safe and controlled setting within which players can learn in an engaging way. Health games for practitioners, such as doctors and nurses, tend to be simulation-based and used for training. This paper is a report on the design and development of the serious game called “Virtual Telemedicine”. The game responds to the need to train doctors for problem-solving in real-life situations of medical care. In the prototype scenario, the user, a practicing doctor, responds to a medical emergency situation to treat the symptoms of a virtual patient by taking specific actions and examining their results based on immediate feedback provided by both the system and the virtual patient. Virtual Telemedicine was created as part of the meducator Best Practice Network. mEducator enables specialized state-of-the-art medical educational content to be discovered, retrieved, shared and re-used across European higher academic institutions. The game makes use of data from an electrocardiogram (ECG), a transthoracic (across the thorax or chest) interpretation of the electrical activity of the heart over a period of time, as detected by electrodes attached to the outer surface of the skin and recorded by a device external to the body. The beta version of the game utilizes variables such as the patient’s heart rate, respiration and temperature, but there is inherent potential to use additional sources of data to increase the complexity of the game. With regard to technical information, the game was created using the programming language C with XNA for the game, C with Flex/Bison for the compiler and C# for the Runner, that runs the scripts. The three pieces are separate and the user runs the first piece. For testing and evaluation purposes, several scripts that used the features of the scripting language were implemented to examine whether they worked properly and a detailed command-by-command debugging was conducted to examine that the flow of the code worked as intended. The educational game “Virtual Telemedicine” is currently working as a standalone application on PCs with the Windows operating system. The game can be shared and repurposed (through changing its scenarios) through mEducator3.0/Melina+, an extended version of Drupal 7, which is offered as an installation profile and enables web site administrators to install a learning management system, which is focused in medical education. As part of future research and development work focusing on the educational evaluation of this serious game, usability testing sessions with practicing doctors will be scheduled to solicit the learners’ feedback and input on the design of the game. Suggested changes will be incorporated in an updated version of the game. A direction for further research is to examine whether learning will be transferred beyond the game context and how, and what is the retention rate of the game when this is used by medical students

Design of evaluation of content sharing solutions in medical education

The problem that the mEducator Best Practice Network (BPN) focuses on refers to the abundance of medical educational content in individual EU academic institutions, which is not widely available or easy to retrieve due to the lack of standardized content sharing mechanisms. The mEducator BPN aims to implement and critically evaluate existing standards and reference models in the field of e-learning in order to enable specialized state-of-the-art medical educational content to be discovered, retrieved, shared and re-used across European institutions. mEducator includes both traditional and user-generated content and addresses several learning contexts ranging from traditional instructional teaching to active learning and experiential teaching/studying approaches. It furthermore includes many different content types, ranging from text to exam sheets, algorithms, teaching files, computer programs (simulators, serious games) and interactive objects (like virtual patients and digital tracings of anatomies), while it covers a variety of tools. The project currently focuses on the development of two contemporary ways of achieving content sharing, mEducator2.0, a solution based on Web2.0 technologies and mEducator3.0, a solution based on Semantic Web Services, and their evaluation. mEducator2.0 develops a brokerage mechanism based on RSS feeds, “mashup” technology and other Web 2.0 technologies that allow the exchange of clinical cases, medical images and videos. mEducator3.0 is a sharing platform using the Semantic Web and its most recent development, the Linked Data approach. Both mEducator solutions are in their beta-version and they are currently being tested by the target user groups of the project such as students, educators, doctors, and health professionals. Their evaluation is based on an evaluation framework for assessing the effectiveness of the platforms that will be implemented through the mEducator BPN and any future platforms that attempt to address the same issues. Our proposed evaluation framework draws upon systems that face similar challenges, such as digital libraries, peer-to-peer and social networks, LCMSs where adaptation issues and courseware reuse are addressed, information retrieval systems, e-learning systems, search engines and public libraries, such as PubMed. It is expected that the resulting solutions will address the needs of different types of users and they will be easily transferable to other domains and disciplines. This paper presents preliminary evaluation results comparing the overall usability of the two solutions and comments on the challenges associated with implementing and carrying out a complex evaluation model

A virtual emergency telemedicine serious game in medical training: A quantitative, professional feedback-informed evaluation study

Background: Serious games involving virtual patients in medical education can provide a controlled setting within which players can learn in an engaging way, while avoiding the risks associated with real patients. Moreover, serious games align with medical students' preferred learning styles. The Virtual Emergency TeleMedicine (VETM) game is a simulation-based game that was developed in collaboration with the mEducator Best Practice network in response to calls to integrate serious games in medical education and training. The VETM game makes use of data from an electrocardiogram to train practicing doctors, nurses, or medical students for problem-solving in real-life clinical scenarios through a telemedicine system and virtual patients. The study responds to two gaps: the limited number of games in emergency cardiology and the lack of evaluations by professionals. Objective: The objective of this study is a quantitative, professional feedback-informed evaluation of one scenario of VETM, involving cardiovascular complications. The study has the following research question: "What are professionals' perceptions of the potential of the Virtual Emergency Telemedicine game for training people involved in the assessment and management of emergency cases?" Methods: The evaluation of the VETM game was conducted with 90 professional ambulance crew nursing personnel specializing in the assessment and management of emergency cases. After collaboratively trying out one VETM scenario, participants individually completed an evaluation of the game (36 questions on a 5-point Likert scale) and provided written and verbal comments. The instrument assessed six dimensions of the game: (1) user interface, (2) difficulty level, (3) feedback, (4) educational value, (5) user engagement, and (6) terminology. Data sources of the study were 90 questionnaires, including written comments from 51 participants, 24 interviews with 55 participants, and 379 log files of their interaction with the game. Results: Overall, the results were positive in all dimensions of the game that were assessed as means ranged from 3.2 to 3.99 out of 5, with user engagement receiving the highest score (mean 3.99, SD 0.87). Users' perceived difficulty level received the lowest score (mean 3.20, SD 0.65), a finding which agrees with the analysis of log files that showed a rather low success rate (20.6%). Even though professionals saw the educational value and usefulness of the tool for pre-hospital emergency training (mean 3.83, SD 1.05), they identified confusing features and provided input for improving them. Conclusions: Overall, the results of the professional feedback-informed evaluation of the game provide a strong indication of its potential as an educational tool for emergency training. Professionals' input will serve to improve the game. Further research will aim to validate VETM, in a randomized pre-test, post-test control group study to examine possible learning gains in participants' problem-solving skills in treating a patient's symptoms in an emergency situation

A virtual emergency telemedicine serious game in medical training: A quantitative, professional feedback-informed evaluation study

Background: Serious games involving virtual patients in medical education can provide a controlled setting within which players can learn in an engaging way, while avoiding the risks associated with real patients. Moreover, serious games align with medical students' preferred learning styles. The Virtual Emergency TeleMedicine (VETM) game is a simulation-based game that was developed in collaboration with the mEducator Best Practice network in response to calls to integrate serious games in medical education and training. The VETM game makes use of data from an electrocardiogram to train practicing doctors, nurses, or medical students for problem-solving in real-life clinical scenarios through a telemedicine system and virtual patients. The study responds to two gaps: the limited number of games in emergency cardiology and the lack of evaluations by professionals. Objective: The objective of this study is a quantitative, professional feedback-informed evaluation of one scenario of VETM, involving cardiovascular complications. The study has the following research question: "What are professionals' perceptions of the potential of the Virtual Emergency Telemedicine game for training people involved in the assessment and management of emergency cases?" Methods: The evaluation of the VETM game was conducted with 90 professional ambulance crew nursing personnel specializing in the assessment and management of emergency cases. After collaboratively trying out one VETM scenario, participants individually completed an evaluation of the game (36 questions on a 5-point Likert scale) and provided written and verbal comments. The instrument assessed six dimensions of the game: (1) user interface, (2) difficulty level, (3) feedback, (4) educational value, (5) user engagement, and (6) terminology. Data sources of the study were 90 questionnaires, including written comments from 51 participants, 24 interviews with 55 participants, and 379 log files of their interaction with the game. Results: Overall, the results were positive in all dimensions of the game that were assessed as means ranged from 3.2 to 3.99 out of 5, with user engagement receiving the highest score (mean 3.99, SD 0.87). Users' perceived difficulty level received the lowest score (mean 3.20, SD 0.65), a finding which agrees with the analysis of log files that showed a rather low success rate (20.6%). Even though professionals saw the educational value and usefulness of the tool for pre-hospital emergency training (mean 3.83, SD 1.05), they identified confusing features and provided input for improving them. Conclusions: Overall, the results of the professional feedback-informed evaluation of the game provide a strong indication of its potential as an educational tool for emergency training. Professionals' input will serve to improve the game. Further research will aim to validate VETM, in a randomized pre-test, post-test control group study to examine possible learning gains in participants' problem-solving skills in treating a patient's symptoms in an emergency situation

Education of Patients With Atrial Fibrillation and Evaluation of the Efficacy of a Mobile Virtual Patient Environment: Protocol for a Multicenter Pseudorandomized Controlled Trial

BackgroundAtrial fibrillation (AF) is the most common sustained cardiac arrhythmia and is a leading cause of mortality and morbidity. Patient knowledge about AF and its management is paramount but often limited. Patients need to be appropriately informed about treatment options, medicinal adherence, and potential consequences of nonadherence, while also understanding treatment goals and expectations from it. Mobile health apps have experienced an explosion both in their availability and acceptance as “soft interventions” for patient engagement and education; however, the prolific nature of such solutions revealed a gap in the evidence base regarding their efficacy and impact. Virtual patients (VPs), interactive computer simulations, have been used as learning activities in modern health care education. VPs demonstrably improved cognitive and behavioral skills; hence, they have been effectively implemented across undergraduate and postgraduate curricula. However, their application in patient education has been rather limited so far. ObjectiveThis work aims to implement and evaluate the efficacy of a mobile-deployed VP regimen for the education and engagement of patients with AF on crucial topics regarding their condition. A mobile VP app is being developed with the goal of each VP being a simple scenario with a set goal and very specific messages and will be subsequently attempted and evaluated. MethodsA mobile VP player app is being developed so as to be used for the design of 3 educational scenarios for AF management. A pseudorandomized controlled trial for the efficacy of VPs is planned to be executed at 3 sites in Greece, Ukraine, and Kazakhstan for patients with AF. The Welch t test will be used to demonstrate the performance of patients’ evaluation of the VP experience. ResultsOur study is at the development stage. A preliminary study regarding the system’s development and feasibility was initiated in December 2022. The results of our study are expected to be available in 2024 or when the needed sample size is achieved. ConclusionsThis study aims to evaluate and demonstrate the first significant evidence for the value of VP resources in outreach and training endeavors for empowering and patients with AF and fostering healthy habits among them. International Registered Report Identifier (IRRID)PRR1-10.2196/4594

Scenario-based assessment for user-testing of medical educational content-sharing solutions

The mEducator Best Practice network implemented standards and reference models in the field of e-learning to develop innovative solutions that enable specialized state-of-the-art medical educational content to be discovered, retrieved, shared and re-used across European institutions. Its target groups include medical students, doctors, educators and health-care professionals. A number of innovative assessment methods, including scenario-based assessment for usability testing complimented with data from online questionnaires, interviews, screen capturing and automated tracking of activity are currently designed and utilized for the evaluation of these solutions. This paper reports on the preliminary findings of usability testing of an instantiation of an mEducator solution (Drupal) conducted by 35 graduate students of an e-health course at an EU university. Usability testing was conducted through scenarios referring to creating, sharing and retrieving medical educational content using mEducator. Implications for designing scenario-based assessment to target different user groups are offered

Medical content searching, retrieving, and sharing over the internet: Lessons learned from the meducator through a scenario-based evaluation

Background: The mEducator Best Practice Network (BPN) implemented and extended standards and reference models in e-learning to develop innovative frameworks as well as solutions that enable specialized state-of-the-art medical educational content to be discovered, retrieved, shared, and re-purposed across European Institutions, targeting medical students, doctors, educators and health care professionals. Scenario-based evaluation for usability testing, complemented with data from online questionnaires and field notes of users' performance, was designed and utilized for the evaluation of these solutions. Objective: The objective of this work is twofold: (1) to describe one instantiation of the mEducator BPN solutions (mEducator3.0-"MEdical Education LINnked Arena" MELINA+) with a focus on the metadata schema used, as well as on other aspects of the system that pertain to usability and acceptance, and (2) to present evaluation results on the suitability of the proposed metadata schema for searching, retrieving, and sharing of medical content and with respect to the overall usability and acceptance of the system from the target users. Methods: A comprehensive evaluation methodology framework was developed and applied to four case studies, which were conducted in four different countries (ie, Greece, Cyprus, Bulgaria and Romania), with a total of 126 participants. In these case studies, scenarios referring to creating, sharing, and retrieving medical educational content using mEducator3.0 were used. The data were collected through two online questionnaires, consisting of 36 closed-ended questions and two open-ended questions that referred to mEducator 3.0 and through the use of field notes during scenario-based evaluations. Results: The main findings of the study showed that even though the informational needs of the mEducator target groups were addressed to a satisfactory extent and the metadata schema supported content creation, sharing, and retrieval from an end-user perspective, users faced difficulties in achieving a shared understanding of the meaning of some metadata fields and in correctly managing the intellectual property rights of repurposed content. Conclusions: The results of this evaluation impact researchers, medical professionals, and designers interested in using similar systems for educational content sharing in medical and other domains. Recommendations on how to improve the search, retrieval, identification, and obtaining of medical resources are provided, by addressing issues of content description metadata, content description procedures, and intellectual property rights for re-purposed content